Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate computed and secret dependencies #123

Merged
merged 7 commits into from
Sep 11, 2023

Conversation

iwahbe
Copy link
Member

@iwahbe iwahbe commented Sep 5, 2023

Fixes #122


This PR separates dependency flow for computedness and secretness.

By default, we apply the following strategies:

Secrets

An output property is secret if and only if there is an input property of the same name that is also secret

Computed

An output property is computed if there is any change in between old and new inputs unless there is an input property of the same name and value that is not computed.

@iwahbe iwahbe self-assigned this Sep 5, 2023
@iwahbe iwahbe marked this pull request as draft September 5, 2023 23:38
@iwahbe iwahbe force-pushed the iwahbe/122/seperate-field-dependencies branch 7 times, most recently from 834ba2d to a19cccb Compare September 6, 2023 19:18
@iwahbe iwahbe marked this pull request as ready for review September 6, 2023 19:27
@iwahbe iwahbe requested a review from a team September 6, 2023 19:27
@iwahbe iwahbe force-pushed the iwahbe/122/seperate-field-dependencies branch from a19cccb to 0cbf9d6 Compare September 6, 2023 21:57
@@ -207,9 +207,16 @@ type OutputField interface {
type InputField interface {
// Seal the interface.
isInputField()

// Only wire secretness, not computedness.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This strikes me as very cryptic, can we unpack a bit?

infer/resource_test.go Outdated Show resolved Hide resolved
} else if !ende.DeepEquals(
r.NewObjectProperty(oldInput),
r.NewObjectProperty(newInput)) {
// If there is a change, then every item item should be
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, why is this? This is simulating preview only I guess but still? It's a little interesting, trying to wrap my head around. So the implementation is allowed to copy oldInput to output, or mark outputs computed.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The default logic is: "if any input property has changed, declare all the output properties as unknown computed, except the outputs that are just pass-through inputs". We mark all output properties as unknown because we don't know if the changed input will change the output. We do know if passthrough inputs are known.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this condition:

if n, ok := newInput[k]; !ok || ende.IsComputed(n) {

If newInput[k] makes ok==false, then there are no asserts performed at all.

So I don't think the property you state in tested here. But implementation might still do it right! It's just not tested seemingly, or I don't get something.

Consider oldInput={"a": 1}, newInput={"a": 42}, output: {"a": 42, "b": "foo"}.

This is trying to simulate "a" input changing, I think this will pass the check although "b": "foo" didn't get marked unknown. By your logic above it seems that "b" should be unknown since we can't assume that pushing changed inputs through the resource black box won't change the outputs.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you're right that the test is missing something. c8c23a0 strengthens the test to assert our expected conditions:

} else if !ende.DeepEquals(
r.NewObjectProperty(oldInput),
r.NewObjectProperty(newInput)) {
// If there is a change, then every item item should be
// computed, except items that mirror a known input.
for k, v := range output {
newV, ok := newInput[k]
if !ok {
assert.True(t, ende.IsComputed(v),
"key: %q", string(k))
} else if !ende.IsComputed(v) {
assert.True(t, ende.DeepEquals(v, newV))
}
}
}

infer/resource_test.go Outdated Show resolved Hide resolved
@t0yv0
Copy link
Member

t0yv0 commented Sep 8, 2023

I'll go read the ticket again #122

@t0yv0
Copy link
Member

t0yv0 commented Sep 8, 2023

The ticket makes sense #122 spec-wise actually I need some more time to work through the PR to grok it.

@t0yv0 t0yv0 self-requested a review September 11, 2023 14:52
@t0yv0
Copy link
Member

t0yv0 commented Sep 11, 2023

Don't want to block the Command P1 fix. I can't find anything wrong with it but I'm still a bit fluffy on the verifying the implementation, if not for the time crunch I'd like to go more thorough on clarifying the by-design behavior especially since secret / unknown inference through black boxes is something we do and want to do across many providers (definitely all bridged providers!) it's an area worth getting rock-solid some time.

@iwahbe
Copy link
Member Author

iwahbe commented Sep 11, 2023

Don't want to block the Command P1 fix. I can't find anything wrong with it but I'm still a bit fluffy on the verifying the implementation, if not for the time crunch I'd like to go more thorough on clarifying the by-design behavior especially since secret / unknown inference through black boxes is something we do and want to do across many providers (definitely all bridged providers!) it's an area worth getting rock-solid some time.

I would love to work with you on nailing the semantics here, and then building a library that works the way we want it to, both here and in the bridge.

@iwahbe iwahbe merged commit ea49c5f into main Sep 11, 2023
5 checks passed
@iwahbe iwahbe deleted the iwahbe/122/seperate-field-dependencies branch September 11, 2023 21:56
iwahbe added a commit that referenced this pull request Sep 21, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Wire secrets separate from dependency flow
2 participants